Multimodal Behavioral Markers Exploring Suicidal Intent in Social Media Videos

Ankit Shah*	Vaibhav*	Vasu Sharma*	Mahmoud Ali Ismail*	Jeffrey Girard	Louis Philippe Morency

Carnegie Mellon University, Language Technologies Institute

* denotes equal contribution

Abstract

Suicide is one of the leading causes of death in the modern world. In this digital age, individuals are increasingly using social media to express themselves and often use these platforms to express suicidal intent. Various studies have inspected suicidal intent behavioral markers in controlled environments but it is still unexplored if such markers will generalize to suicidal intent expressed on social media. In this work, we set out to study multimodal behavioral markers related to suicidal intent when expressed on social media videos. We explore verbal, acoustic and visual behavioral markers in the context of identifying individuals at higher risk of suicidal attempt. %and provide them with timely intervention and help. Our analysis reveals that frequent silences, slouched shoulders, rapid hand movements and profanity are predominant multimodal behavioral markers indicative of suicidal intent\footnote{Code: \url{https://github.com/ankitshah009/icmi_19_suicidal_intent_detection}}.

Network Architecture

Paper

Citation   Ankit Shah*, Vaibhav*, Vasu Sharma*, Mahmoud Al Ismail*, Jeffrey Girard, Louis Philippe Morency Multimodal Behavioral Markers Exploring Suicidal Intent in Social Media Videos Accepted at 21st ACM International Conference on Multimodal Interaction (ICMI) 2019. [Show BibTex] [PDF] [Code - GitHub] @inproceedings{Shah:2019:MBM:3340555.3353718, author = {Shah, Ankit Parag and Sharma, Vasu and Vaibhav, Vaibhav and Alismail, Mahmoud and Morency, Louis-Philippe}, title = {Multimodal Behavioral Markers Exploring Suicidal Intent in Social Media Videos}, booktitle = {2019 International Conference on Multimodal Interaction}, series = {ICMI '19}, year = {2019}, isbn = {978-1-4503-6860-5}, location = {Suzhou, China}, pages = {409--413}, numpages = {5}, url = {http://doi.acm.org/10.1145/3340555.3353718}, doi = {10.1145/3340555.3353718}, acmid = {3353718}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {Multimodal behavior markers, healthcare analysis, multimodal predictive modelling, social media, suicidal ideation}, }

Related Papers

Anurag Kumar, Bhiksha Raj Weakly Supervised Scalable Audio Content Analysis IEEE International Conference on Multimedia and Expo (ICME), July 2016.

Romain Serizel, Nicolas Turpault, Hamid Eghbal-Zadeh, Ankit Parag Shah Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments DCASE2018 Workshop, Nov 2018

Other works on related to “Audio Event” Learning

Szu-Yu Chou, Jyh-Shing Roger Jang, Yi-Hsuan Yang Learning to Recognize Transient Sound Events Using Attentional Supervision 27th International Joint Conference on Artificial Intelligence, 2018

Shizhe Chen, Jia Chen, Qin Jin, Alexander Hauptmann Class-aware Self-Attention for Audio Event Recognition International Conference on Multimedia Retrieval, 2018

Funding

This research was supported by: Carnegie Mellon University

Contact

For questions/comments, contact Vaibhav Vasu Sharma Ankit Shah Mahmoud Al Ismail <>